Text representation method and apparatus

ABSTRACT

The present invention relates to text analysis, and discloses a text representation method. Aspects include identifying concepts in the text by using a knowledge base and determining relationship between the concepts and generating a concept graph by using the relationship between the concepts. Aspects also include determining connected components of the concept graph; calculating weight of the connected components and determining the concepts representing the text according to the weight of the connected components. By using correlation between concepts in a knowledge base and according to connected component theory of a graph, finds out a set of concepts which best represents subject of the text, and removes concepts irrelevant to the subject, thus improving accuracy of text representation and reducing noise.

FOREIGN PRIORITY

This application claims priority to Chinese Patent Application No.201410705205.X; filed Nov. 28, 2014, and all the benefits accruingtherefrom under 35 U.S.C. § 119, the contents of which in its entiretyare herein incorporated by reference.

BACKGROUND

The present invention relates to text analysis, and more particularly,to a text representation method and apparatus.

Text analysis has wide application in fields such as informationretrieval, data mining, and machine translation. Text analysis refers toextracting representation Of text and its feature items, and convertingunstructured original text into structured information which can beidentified and processed by a computer, i.e., performing scientificabstraction on text and establishing its mathematical model to describeand replace the text, such that the computer can realize textidentification by computing and operating such a model.

Latent semantic analysis (LSA), also known as latent semantic index(LSI), is a known index and retrieval method. This method, liketraditional vector space model, uses vectors to represent terms anddocuments, and determines relationship between terms and documentsthrough relationship between vectors (e.g., angles); the difference liesin that, LSA maps terms and documents to a latent semantic space, thusremoving some “noises” in original vector space and improving accuracyin information retrieval. However, LSA still does not solve the problemof polysemy, and only solves the problem of synonym. Because LSArepresents each term as a point in latent semantic space, the pluralityof meanings of one term correspond to one point in the space and are notdistinguished.

The intention of ESA (Explicit Semantic Analysis) is to for a givendocument segment, ESA will generate a semantic interpreter, which canproject this segment to some related wild concepts and perform sortingaccording to degree of relevancy. The method of ESA determines a set ofconcepts by only considering similarity between context of the conceptsand the text, and does not consider coherence among the concepts.

SUMMARY

In one aspect of the present invention, there is provided a textrepresentation method. The method includes identifying concepts in thetext by using a knowledge base and determining relationship between theconcepts; and generating a concept graph by using the relationshipbetween the concepts. The method also includes determining connectedcomponents of the concept graph, calculating weight of the connectedcomponents, and determining the concepts representing the text accordingto the weight of the connected components.

In another aspect of the present invention, there is provided a textrepresentation apparatus. The apparatus includes a concept identifyingmodule configured to identify concepts in the text by using a knowledgebase and determine relationship between the concepts and a concept graphgenerating module configured to generate a concept graph by using therelationship between the concepts. The apparatus also includes aconnected component determining module configured to determine connectedcomponents of the concept graph, a weight calculating module configuredto calculate weight of the connected components, and a conceptdetermining module configured to determine the concepts representing thetext according to the weight of the connected components. By usingcorrelation between concepts in a knowledge base and according toconnected component theory of a graph, finds out a set of concepts whichbest represents subject of the text, and removes concepts irrelevant tothe subject, thus improving accuracy of text representation and reducingnoise.

BRIEF DESCRIPTION OF THE DRAWINGS

Through the more detailed description of some embodiments of the presentdisclosure in the accompanying drawings, the above and other objects,features and advantages of the present disclosure will become moreapparent, wherein the same reference generally refers to the samecomponents in the embodiments of the present disclosure.

FIG. 1 shows a block diagram of an exemplary computer system which isapplicable to implement the embodiments of the present invention;

FIG. 2 shows a text representation method according to an embodiment ofthe present invention;

FIG. 3 shows an undirected graph M;

FIG. 4 shows two connected components H₁ and H₂ of the undirected graphM of FIG. 3;

FIG. 5 shows an example of a piece of text according to an embodiment ofthe present invention;

FIG. 6 shows an association graph between concepts in the text exampleof FIG. 5 and other concepts in a knowledge base;

FIG. 7 shows a concept graph generated according to relationship betweenthe concepts identified in the text example; and

FIG. 8 shows a text representation apparatus 800 according to anembodiment of the present invention.

DETAILED DESCRIPTION

Some preferable embodiments will be described in more detail withreference to the accompanying drawings, in which the preferableembodiments of the present disclosure have been illustrated. However,the present disclosure can be implemented in various manners, and thusshould not be construed to be limited to the embodiments disclosedherein. On the contrary, those embodiments are provided for the thoroughand complete understanding of the present disclosure, and completelyconveying the scope of the present disclosure to those skilled in theart.

Referring now to FIG. 1, in which a block diagram of an exemplarycomputer system/server 12 which is applicable to implement theembodiments of the present invention is shown. Computer system/server 12is only illustrative and is not intended to suggest any limitation as tothe scope of use or functionality of embodiments of the inventiondescribed herein.

As shown in FIG. 1, computer system/server 12 is shown in the form of ageneral-purpose computing device. The components of computersystem/server 12 may include, but are not limited to, one or moreprocessors or processing units 16, a system memory 28, and a bus 18 thatcouples various system components including system memory 28 toprocessor 16.

Bus 18 represents one or more of any of several types of bus structures,including a memory bus or memory controller, a peripheral bus, anaccelerated graphics port, and a processor or local bus using any of avariety of bus architectures. By way of example, and not limitation,such architectures include Industry Standard Architecture (ISA) bus,Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, VideoElectronics Standards Association (VESA) local bus, and PeripheralComponent Interconnect (PCI) bus.

Computer system/server 12 typically includes a variety of computersystem readable media. Such media may be any available media that isaccessible by computer system/server 12, and it includes both volatileand non-volatile media, removable and non-removable media.

System memory 28 can include computer system readable media in the formof volatile memory, such as random access memory (RAM) 30 and/or cachememory 32. Computer system/server 12 may further include otherremovable/non-removable, volatile/non-volatile computer system storagemedia. By way of example only, storage system 34 can be provided forreading from and writing to a non-removable, non-volatile magnetic media(not shown and typically called a “hard drive”). Although not shown, amagnetic disk drive for reading from and writing to a removable,non-volatile magnetic disk (e.g., a “floppy disk”), and an optical diskdrive for reading from or writing to a removable, non-volatile opticaldisk such as a CD-ROM, DVD-ROM or other optical media can be provided.In such instances, each can be connected to bus 18 by one or more datamedia interfaces. As will be further depicted and described below,memory 28 may include at least one program product having a set (e.g.,at least one) of program modules that are configured to carry out thefunctions of embodiments of the invention.

Program/utility 40, having a set (at least one) of program modules 42,may be stored in memory 28 by way of example, and not limitation, aswell as an operating system, one or more application programs, otherprogram modules, and program data. Each of the operating system, one ormore application programs, other program modules, and program data orsome combination thereof, may include an implementation of a networkingenvironment. Program modules 42 generally carry out the functions and/ormethodologies of embodiments of the invention as described herein.

Computer system/server 12 may also communicate with one or more externaldevices 14 such as a keyboard, a pointing device, a display 24, etc.;one or more devices that enable a user to interact with computersystem/server 12; and/or any devices (e.g., network card, modem, etc.)that enable computer system/server 12 to communicate with one or moreother computing devices. Such communication can occur via Input/Output(I/O) interfaces 22. Still yet, computer system/server 12 cancommunicate with one or more networks such as a local area network(LAN), a general wide area network (WAN), and/or a public network (e.g.,the Internet) via network adapter 20. As depicted, network adapter 20communicates with the other components of computer system/server 12 viabus 18. It should be understood that although not shown, other hardwareand/or software components could be used in conjunction with computersystem/server 12. Examples, include, but are not limited to: microcode,device drivers, redundant processing units, and external disk drivearrays, RAID systems, tape drives, and data archival storage systems,etc.

FIG. 2 shows a text representation method according to an embodiment ofthe present invention, comprising: at step S201, identifying concepts inthe text by using a knowledge base and determining relationship betweenthe concepts; at step S202, generating a concept graph by using therelationship between the concepts; at step S203, determining connectedcomponents of the concept graph; at step S204, calculating weight of theconnected components; at step S205, determining the conceptsrepresenting the text according to the weight of the connectedcomponents.

The knowledge base according to an embodiment of the present inventiondefines concepts and relationship between concepts that have broadcontents. The knowledge base is described by RDF (resource descriptionframework), and basic idea of RDF is: (1) collectively referring all thethings (specific or abstract, existing or non-existing) which can beidentified on Web as “resource”; (2) identifying resource by URI(Universal Resource Identifier); (3) describing features of resourcesand relationship between the resources by properties. Concept in theknowledge base in embodiments of the present invention is somethingwhich possesses distinctiveness and can exist independently, andcorresponds to the resources in the RDF identified by URI; for example,currently DBpedia is a relatively typical and mature knowledge base.

In RDF, basic structure of any expression is a set of triples, eachtriple is comprised of subject, predicate and object. The subjectcorresponds to resource, is anything that may possess an URI, e.g.,http://dbpedia.org/resource/China; the predicate corresponds toproperties of the resource, e.g., author, firstname; the objectcorresponds to value of a property, it may be a character string oranother resource, e.g., David orhttp://dbpeida.org/resource/United_States. It is defined that S is thesubject, P is the predicative, and O is the object, then a RDF triplemay be expressed as: (S, P, O). In embodiments of the present invention,RDF is used to describe relationship between concepts in a knowledgebase, and association between the concepts of the knowledge base may bedescribed by a directional graph. Nodes in the directional graph denoteconcepts, and directional edges between the nodes denote associationrelationship between the concepts.

At step S201, identifying concepts in the text by using a knowledge baseand determining relationship between the concepts. First, all possiblenames of the concepts are extracted from the knowledge base to form aname dictionary, and the mapping relationship between the concepts andthe names is established. Names of the concepts are extracted by usingsome properties related to the names in the knowledge base, andproperties related to names are looked up in predicative of the RDFtriad, e.g., “name”, “fullname”, “nickname”, and the found subjectscorresponding to the properties related to the names are used asconcepts, and value of properties in the corresponding objective areused as names of concepts, thus establishing a mapping table between theconcepts and the names. Similarly, the property “redirect” also givespossible names of the concepts. One concept may correspond to multiplenames, and one name may also correspond to multiple concepts. Table 1illustratively shows a mapping table of some of the names extracted fromthe knowledge base abc and the concepts.

TABLE 1 Name Concept Physical educationhttp://abc.org/resource/Physical_education PE parents human_Parentshttp://abc.org/resource/Parent aerobic exercise aerobic_workouthttp://abc.org/resource/Aerobic_exercise cardio_training ageaging_process http://abc.org/resource/Aging aged_population educationhttp://abc.org/resource/Education educational systemhttp://abc.org/resource/Master_of_Educationhttp://abc.org/resource/Local_education_authority

After obtaining the name dictionary, given a piece of text, all names inthe text that appear in the name dictionary may be identified, and atleast one concept corresponding to the identified names is determined byusing the mapping relationship. For the case that one name correspondsto multiple concepts, semantic disambiguation should be performed, andconcept having the same meaning as the identified name is determinedfrom at least one concept according to semantic analysis on context ofthe text. Semantic disambiguation performs scoring by combining avariety of features, and the concept with the highest score is selectedas the concept corresponding to the name. Here the features may includecomparing similarity of the name in context of the text and in contextof the concept, or obtaining, by collecting statistics, the concepthaving the largest probability to correspond to the name in corpus ofthe knowledge base. Those skilled in the art will appreciate that, inaddition to the above methods, there may also be many methods to realizesemantic disambiguation, as long as the concept corresponding to thenames in the text can be determined accurately and uniquely, and thepresent invention has no limitation thereto.

At step S202, generating a concept graph by using the relationshipbetween the concepts, wherein the concept graph V(G) is an undirectedgraph, and nodes therein denote concepts, and the path between any twonodes V_(i) and V_(j) indicates that in the knowledge base, the conceptcorresponding to node V_(i) is associated with the concept correspondingto node V_(j), and the path does not have direction; if node V_(i) isdirectly connected to V_(j) (there is no other node between the twonodes), the path between nodes V_(i) and V_(j) corresponds to an edgebetween nodes V_(i) and V_(j); if there is other node between nodesV_(i) and V_(j), then the path between nodes V_(i) and V_(j) is formedby connecting the paths between every two adjacent nodes between nodesV_(i) and V_(j).

At step S203, determining connected components of the concept graph, andthe maximum connected sub-graph of the undirected graph G is referred asconnected component of G, and any connected graph has only one connectedcomponent, i.e., itself; in the undirected graph G, if any two differentnodes V_(i) and V_(j) are connected, then G is called a connected graph.A non-connected unidirectional graph has a plurality of connectedcomponents. FIG. 3 shows an undirected graph M, which is a non-connectedgraph including four nodes V₁, V₂, V₃ and V₄; FIG. 4 shows two connectedcomponents H₁ and H₂ of the undirected graph M of FIG. 3.

At step S204, calculating weight of the connected components, and weightof the connected components are calculated according to node weight ofthe connected components and path weight between the nodes. According toan embodiment of the present invention, the node weight may bedetermined according to number of times that the concepts correspondingto the nodes appear in the text. According to an embodiment of thepresent invention, the path weight between the nodes may be determinedaccording to length of path between the nodes representing two conceptsin the knowledge base, e.g., the connected components H₁ in the FIG. 4;node V₁ and V₂ are connected directly (there is no other nodetherebetween); the path L(V₁V₂) between nodes V₁ and V₂ is formed by theedge between nodes V₁ and V₂, indicating that the concept correspondingto node V₁ is associated with the concept corresponding to node V₂.Nodes V₂ and V₃ are connected directly, and the path L(V₂V₃) betweennodes V₂ and V₃ is formed by the edge between nodes V₂ and V₃,indicating that the concept corresponding to node V₂ is associated withthe concept corresponding to node V₃. The length of path L(V₁V₂) betweennodes V₁ and V₂ is set to one unit length, and nodes V₁ and V₃ areconnected through node V₂, indicating the concept corresponding to nodeV₁ is associated with the concept corresponding to node V₃ through theconcept corresponding to node V₂, and path between nodes V₁ and V₃ isformed by connecting paths L(V₁V₂) and L(V₂V₃), and length of the pathL(V₁V₃) between nodes V₁ and V₃ is set to two unit length.

Taking the connected component H₁ in FIG. 3 for example, statisticsabout the following are collected: number of times T₁ that concept C₁corresponding to node V₁ appears in the text, number of times T₂ thatconcept C₂ corresponding to node V₂ appears in the text, number of timesT₃ that concept C₃ corresponding to node V₃ appears in the text, andnumber of times T₄ that concept C₄ corresponding to node V₄ appears inthe text, and node weight of the connected component H₁ isW(V)=T₁+T₂+T₃+T₄. In the knowledge base, length of the path betweenconcept C₁ and concept C₂ is K₁, length of the path between concept C₂and C₃ is K₂, length of the path between concept C₃ and C₄ is K₃, lengthof the path between concept C₄ and C₁ is K₄, and path weight of theconnected component H₁ is;

${{W(K)} = {\frac{1}{K_{1}} + \frac{1}{K_{2}} + \frac{1}{K_{3}} + \frac{1}{K_{4}}}},$according to an embodiment of the present invention, weight W(H₁) of theconnected component H₁ is calculated as follows:

${W\left( H_{1} \right)} = {{{W(V)} + {W(K)}} = {\left( {T_{1} + T_{2} + T_{3} + T_{4}} \right) + \left( {\frac{1}{K_{1}} + \frac{1}{K_{2}} + \frac{1}{K_{3}} + \frac{1}{K_{4}}} \right)}}$

At step S205, determining the concepts representing the text accordingto the weight of the connected components. According to an embodiment ofthe present invention, the concepts representing the text may bedetermined according to a Top-N pattern. Specifically, weight of aplurality of connected components are sorted in descending order, andthe first N connected components are selected; the text is representedby the concepts corresponding to the nodes contained in the first Nconnected components, wherein N may be specified in advance based onexperience. According to an embodiment of the present invention, theconnected component with the maximum weight may be selected, and thetext is represented by the concept corresponding to the node containedin the connected component with the maximum weight.

For a paragraph of text, a set of concepts that may represent the textall involve similar subjects, and the concepts are closely related. Themethod according to embodiments of the present invention, by usingcorrelation between concepts in a knowledge base, maps names identifiedfrom the text to a concept space of the knowledge base, and according toconnected component theory of a graph, finds out a set of concepts whichbest represents subject of the text, and removes concepts irrelevant tothe subject, thus improving accuracy of text representation and reducingnoise.

FIG. 5 shows an example of a piece of text according to an embodiment ofthe present invention, names in the text example are identified by usingtable 1, i.e., Physical education, PE, education, parents, aerobicexercise and age, and the identified names are mapped to the conceptsrepresented by URIs according to the mapping relationship between thenames and concepts, for example, Physical education, PE corresponds tothe concept Physical_education in knowledge base abc,http://abc.org/resource/Physical_education, parents corresponds toconcept Parent in knowledge base abc, http://abc.org/resource/Parent,aerobic_exercise corresponds to concept Aerobic_exercise in knowledgebase abc, http://abc.org/resource/Aerobic_exercise, age corresponds toconcept Aging in knowledge base abc: http://abc.org/resource/Aging,education corresponds to the concepts in knowledge base abc:

-   -   http://abc.org/resource/Education;    -   http://abc.org/resource/Master_of_Education; and    -   http://abc.org/resource/Local_education_authority.

It is uniquely determined that http://abc.org/resource/Education is theconcept corresponding to education in the text example through the abovedescribed semantic disambiguation method in conjunction with thecontext.

FIG. 6 shows an association graph between concepts in the text exampleof FIG. 5 and other concepts in a knowledge base.

Node C₁ corresponds to concept Category:physical_exercise in theknowledge base, http://abc.org/ontopology/Category:physical_exercise;Node C₂ corresponds to concept Category:Physical_education in theknowledge base, http://abc.org/ontopology/Category:Physical_education;Node C₃ corresponds to concept Category:education_by_subject in theknowledge base, http://abc.org/ontopology/Category:education_by_subject;Node C₄ corresponds to concept Category:education in the knowledge base,http://abc.org/ontopology/Category:education; Node C₅ corresponds toconcept Education in the knowledge base,http://abc.org/resource/Education;

Node C₆ corresponds to concept Aerobic_exercise in the knowledge base,http://abc.org/resource/Aerobic_exercise; Node C₇ corresponds to conceptPhysical_education in the knowledge base,http://abc.org/resource/Physical_education; Node C₈ corresponds toconcept Category: aerobic_exercise in the knowledge base,http://abc.org/resource/Category:aerobic_exercise; Node C₉ correspondsto concept Category:Disease in the knowledge base,http://abc.org/resource/Category:Disease; and Node C₁₀ corresponds toconcept Aging in the knowledge base, http://abc.org/resource/Aging; andNode C₁₁ corresponds to concept Parent in the knowledge base,http://abc.org/resource/Parent.

As shown in FIG. 6, concept Physical_education is associated withconcept Education through concepts Category:physical_education,Category:education_by_subject and Category:education, and length of thepath between concepts Physical_education and Aerobic_exercise is 4 unitsof length; concept physical_education is associated with conceptAerobic_exercise through concepts Category:physical_exercise andCategory: aerobic_exercise, and length of the path between conceptsPhysical_education and Aerobic_exercise is 3 units of length; conceptAging is associated with concept Category:Disease. Concept associatedwith concept Parent is not found in the knowledge base.

FIG. 7 shows a concept graph generated according to relationship betweenthe concepts identified in the text example. Since conceptPhysical_education is associated with concept Education, a pathconnection will be generated between the two concepts. Since conceptphysical_education is associated with concept Aerobic_exercise, a pathconnection will be generated between the two concepts. Concepts Agingand Parent do not have other concept associated therewith, thus they areindependent nodes. According to definition of connected component, thegenerated concept graph has 3 connected components, that is, H₁, H₂ andH₃ respectively.

Statistics about the following are collected: number of times thatconcept Physical_education appears in the text is 3, number of timesthat concept Aerobic_exercise appears in the text is 1, number of timesthat concept Education appears in the text is 1; FIG. 6 shows thatlength of the path between concepts Physical_education and Education is4 units of length, and length of the path between conceptsPhysical_education and Aerobic_exercise is 3 units of length, and weightW(H₁) of the connected component H₁ is calculated as follows:

${W\left( H_{1} \right)} = {{\left( {3 + 1 + 1} \right) + \left( {{1/4} + {1/3}} \right)} = {5\frac{7}{12}}}$

Both the number of times that concepts Aging and Parent appear in thetext is 1, and there is no other concept associated therewith, and thereis no path to other nodes.W(H ₂)=1W(H ₃)=1

According to an embodiment of the present invention, the conceptsPhysical_education, Aerobic_exercise and Education contained in theconnected component H₁ having the maximum weight are selected torepresent the text example shown in FIG. 5.

Various embodiments for implementing the method of the present inventionhave been described above with reference to accompanying drawings. Thoseskilled in the art will appreciate that, the above method may beimplemented in software, hardware, or a combination thereof. Inaddition, those skilled in the art will appreciate that, by implementingsteps in the above method in software, hardware or a combinationthereof, a text representation apparatus may be provided. Although theapparatus is the same as a general-purpose processing apparatus inhardware structure, due to function of the software contained therein,the apparatus presents a characteristic distinct from thegeneral-purpose processing apparatus, thus forming the apparatusaccording to embodiments of the present invention.

Based on a same inventive concept, there is also provided a textrepresentation apparatus according to an embodiment of the presentinvention. FIG. 8 shows a text representation apparatus 800 according toan embodiment of the present invention, comprising: a conceptidentifying module 801 configured to identify concepts in the text byusing a knowledge base and determine relationship between the concepts;a concept graph generating module 802 configured to generate a conceptgraph by using the relationship between the concepts; a connectedcomponent determining module 803 configured to determine connectedcomponents of the concept graph; a weight calculating module 804configured to calculate weight of the connected components; a conceptdetermining module 805 configured to determine the concepts representingthe text according to the weight of the connected components.

According to an embodiment of the present invention, wherein theknowledge base describes relationship between concepts by using aresource description language. According to an embodiment of the presentinvention, the apparatus further comprising: a concept extracting moduleconfigured to extract all possible names of the concepts from theknowledge base and form a name dictionary; a mapping establishing moduleconfigured to establish mapping between all the possible names in thename dictionary and the concepts; a name identifying module configuredto identify all the names in the text that appear in the namedictionary; wherein the concept identifying module 801 is configured todetermine at least one concept corresponding to the identified name byusing the mapping, and determine concepts having same meaning as theidentified names from the at least one concept according to semanticanalysis on context of the text.

According to an embodiment of the present invention, wherein nodes ofthe concept graph correspond to concepts in the text, and paths betweenthe nodes indicate association relationship of concepts in the textcorresponding to the nodes.

According to an embodiment of the present invention, wherein weight ofthe connected components is calculated according to node weight of theconnected components and path weight between the nodes.

According to an embodiment of the present invention, wherein the nodeweight of the connected components is determined according to number oftimes that the concepts corresponding to the nodes appear in the text,and the path weight between the nodes of the connected components isdetermined according to length of path between the nodes of theconnected components in a directed graph describing association betweenthe concepts of the knowledge base.

According to an embodiment of the present invention, wherein the conceptdetermining module is further configured to represent the text withconcepts corresponding to nodes contained in a connected componenthaving the maximum weight.

For specific implementation of each of the above modules, reference maybe made to the detailed description of the text representation methodaccording to embodiments of the present invention, the description ofwhich will be omitted here.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

What is claimed is:
 1. A text representation apparatus, comprising: oneor more processors; a memory, said memory coupled to the one or moreprocessors; a concept identifying module coupled to the one or moreprocessors and configured to identify concepts in a text by using aknowledge base and determine relationship between the concepts; whereinconcepts are identified by matching names in the text with respectivesame names appearing in a table comprising names associated withconcepts, and wherein each concept in the table is associated with auniform resource identifier; a concept graph generating module coupledto the one or more processors and configured to generate a concept graphby using the relationship between the concepts; a connected componentdetermining module coupled to the one or more processors and configuredto determine connected components of the concept graph, wherein nodes ofthe concept graph correspond to respective names in the text, each namehaving an associated concept defined by the table, and paths between thenodes indicate a relationship between concepts corresponding to thenodes; a weight calculating module coupled to the one or more processorsand configured to calculate a weight of the connected components,wherein the weight is calculated by the following”W=((T1)+(T2)+(T3))+((1/K1)+(1/K2)), wherein W is the weight, T1 is afrequency that a first name appears in the text, T2 is a frequency thata second name appears in the text, T3 is a frequency that a third nameappears in the text, K1 is a path length is between a concept associatedthe first name and a concept associated with the second name, and K2 isa path length is between a concept associated the first name and aconcept associated with the third name; a concept determining modulecoupled to the one or more processors and configured to determine theconcepts representing the text according to a top-N pattern based on theweight of the connected components, wherein the concept determiningmodule removes concepts not relevant to a subject associated with thetext and represents the text with concepts corresponding to one or morenodes contained in a connected component having a maximum weight.
 2. Theapparatus according to claim 1, further comprising: a concept extractingmodule configured to extract all possible names of the concepts from theknowledge base and form a name dictionary; a mapping establishing moduleconfigured to establish mapping between all the possible names in thename dictionary and the concepts; a name identifying module configuredto identify all the names in the text that appear in the namedictionary; wherein the concept identifying module is configured todetermine, for each identified name, at least one concept correspondingto the identified name by using the mapping, and determine, for eachidentified name, concepts having same meaning as the identified namefrom the at least one concept according to semantic analysis on contextof the text.
 3. The apparatus according to claim 1, wherein weight ofthe connected components is calculated according to node weight of theconnected components and path weight between the nodes.
 4. The apparatusaccording to claim 1, wherein the knowledge base describes relationshipbetween concepts by using a resource description language.